Accuracy of Tutors’ Assessments of their Students by Tutoring Context

In this study we compared the accuracy of tutors’ assessments of their students’ general competence, conceptual knowledge and affective state in two different tutoring contexts: face-to-face (FTF) and computer-mediated (CM). We found that the accuracy of tutors’ assessments of their students was dependent on both the type of student information that was assessed, and, to a lesser extent, the tutoring context . Only tutors’ assessments of their students’ general competence, as opposed to their assessments of their students’ individual conceptual knowledge or their students’ motivation, was affected by the manipulations. Introduction One-to-one human tutoring is generally more effective than classroom instruction (e.g., Bloom, 1984; Cohen, Kulik & Kulik, 1982). Many have assumed that this is in part because tutors can understand their students’ domain competence and attitudes and, through this understanding, adapt their instruction to their students. However, some (e.g., Chi, 1996) have questioned this assumption. As part of a larger study testing this adaptive hypothesis, we sought a way to vary tutors’ understanding of their students while minimizing disruption to the tutoring situation. In this study, we varied the amount of experience tutors had with a particular student. In some conditions, each tutor tutored the same student for the entirety of the tutoring session (Same conditions); in other conditions, each tutor tutored four different students (one at a time) in the tutoring session (Different conditions). Thus, tutors in the Same conditions had more experience with an individual student than tutors in the Different conditions. As the results will show, this manipulation did make a difference in the accuracy of some types of tutors’ assessments of their students, demonstrating that tutors can assess their individual students during tutoring. A second manipulation was tutoring context. We compared tutoring in a spoken face-to-face (FTF) context with tutoring in a computer-mediated (CM) context, in which tutors and students could not see or hear each other but communicated through typed messages , to determine the effects of the tutoring context on the tutors’ assessments of their students ’ domain competence, conceptual knowledge, and affective states. Earlier work comparing CM to FTF communication contrasts the amount and content of message production (e.g., Lebie, Rhoades & McGrath, 1996; Ruberg, Moore & Taylor, 1996) and efficiency of the message (e.g., Hausmann & Chi, 2002); however, no work that we are aware of has been done comparing the accuracy of tutors’ assessments of their students as a function of tutoring context. In FTF communication, more sources of information are available to tutors that they may use to assess their students. For example, in a FTF context, prosodic information (e.g., vocal pitch, loudness, turn duration, speaking rate) is available that is not available in a text only CM context (Litman, 2002). Other types of potentially useful information available only in the FTF context include facial expressions (e.g., puzzled, upset, disinterested), body language (e.g., leaning direction), and nonlinguistic verbalizations (Fox, 1993). All of these may be particularly useful sources of information about the student’s affective state. However, there are other sources of information available in a FTF context that may not be useful (e.g., the student’s general appearance), or may even impair the tutor’s ability to accurately assess the student. Although tutors in a FTF context have more information available to them, there are some aspects of CM tutoring that may benefit the development of tutors’ assessments of their students. For example, in CM tutoring, there is a record of the dialog, allowing tutors to re-read portions of the current dialog they may have initially missed or mis understood. In a FTF condition, if tutors do not hear or understand messages from their students when they are spoken, this information may be lost to the tutor. Additionally, tutors in CM tutoring may refer back to dialog previously read. Because of repeated exposure to this information, the likelihood of retaining that information increases. In FTF tutoring, tutors must rely on their memories of the past discussion or on notes taken (Lebie et al., 1996). Additionally, in CM tutoring, there is more time between conversational turns (Clark & Brennan, 1996), allowing more opportunity for tutors to think about and more deeply process information. Our research addressed the question of whether tutors develop more accurate assessments of their students’ general domain competence, conceptual knowledge, and affective state in a CM context, a FTF context, or whether